Unidata Outreach Accomplishments and Challenges

Ben Domenico, August 2014

Relationship to Current Unidata Strategic Plan and Proposed Work

Below are a few excerpts from the current Unidata Strategic Plan that highlight the importance of the outreach activities summarized in this status update?

  • ... to build infrastructure that makes it easy to integrate and use data from disparate geoscience disciplines

  • Data formats like netCDF, together with community-based data standards like the Climate and Forecast metadata convention and the Common Data Model are enhancing the widespread usability and interoperability of scientific datasets.

  • Advance geoscience data and metadata standards and conventions

  • ... our experience shows us that robust solutions arise from community and collaborative efforts

  • ... close partnerships and collaboration with geoscience data providers, tool developers, and other stakeholders, and the informed guidance of our governing committees 

Summary of Recent Progress and Near-term Goals

Cloud-based Collaborative Python Development

As part of Unidata's first training workshop on software development using Python, experimentation began with the Wakari cloud-hosted development environment.   This work has continued since then but at a slower pace due to the departure of software engineer, Marcus Hermida.   Since then, the work has been focused on the use of the community supported OWSlib tools for accessing data from OPeNDAP servers via a brokering layer that makes the data available via other standard interfaces, especially Web Map Service (WMS) and Web Coverage Serviced (WCS).   If time allows, it will be good to fold in the work on ncSOS into this effort.

This work will continue as resources allow.  To be honest, this project has been on the back burner recently.

Progress on OGC standardization of CF-netCDF

As the official UCAR representative to the OGC Technical Committee, Unidata participates in 3-4 technical committee meetings per year to ensure that Unidata and UCAR needs are met in the emerging international standards.

Since the March 2014 report, Unidata has negotiated for UCAR a renewed five year Memo of Understanding with the Open Geospatial Consortium (OGC) which provides voting membership for UCAR on the OGC Technical Committee.   Unidata has also agreed to host a set OGC Technical Committee meetings in June 2015.

The CF-netCDF Standards Working Group (SWG) has been dissolved and replaced by the NetCDF SWG.  
This reflects the fact that some of the conventions extensions under consideration (e.g., netCDF-uncertainty) are not yet CF conventions.  A new OGC NetCDF Core and Extensions Primer has been written to incorporate the change. The OGC-adopted standards documents are available athttp://www.opengeospatial.org/standards/netcdf
The primary focus of the NetCDF SWG now is:
  • NcML specification and NcML-Gml (draft discussion paper has been submitted)
  • Extension to OWS Common for CF-netCDF data exchange
  • Extension to WCS 2.0 core DescribeCoverage  for CF-netCDF data description
  • NetCDF-Uncertainty.  Focus on netcdf-u proposal for this discussion.   Broader discussion taking place in other OGC venues (ESS, Coverages, UncertML andCF communities) as well.
  • How does WCS2.0 Extension for Met Data fit with proposed OWS Common extension and WCS DescribeCoverage
  • NetCDF Search Conceptual Model
  • Conventions for netCDF4 enhanced data model (e.g., use of hierarchical groups and user-defined data types)

ODIP

The ODIP (Ocean Data Interoperability Platform) is entering the third and final year of the proposed work.  Unidata's technologies (especially THREDDS and netCDF) are part of the project and we also maintain a liaison role and serve on the steering team.   Unidata has participated in the ODIP workshops and steering team meetings and is helping to develop a proposal for a second phase of the project.  More details on ODIP, the workshops and plans can be found at:

          http://www.odip.org/

CUAHSI Hydrological Information Systems (HIS) Standing Committee

The CUAHSI HIS Standing Committee is roughly equivalent to the Unidata Strategic Advisory Committee (USAC).   Jeff Weber and Ben Domenico participated in several teleconferences for this group and Ben attended the face to face meeting last month.   As Unidata's Policy Committee in the past and more recently with USAC, there are difficult decisions about what to focus on in a era of limited resources.

EarthCube

Discrete Continuous Building Block Project.

Unidata continues an active role in the Discrete Continuous Building Block initiative led by the University of Texas.

From the project description:

Geoscience information is defined on both discrete and continuous spatial domains.  Discrete spatial domains include point locations of observations at measurement sites and GIS coverages of point, line and area features used for observation and data interpretation.  Continuous spatial domains are used in geophysical fluid sciences such as for the atmosphere, oceans, and land subsurface to describe arrays of measured or modeled variables defined on a mesh of uniformly spaced points.  Data defined on either discrete or continuous spatial domains may also vary discretely or continuously in time, ranging from one-time samples, to samples at random points of time, to samples at regularly spaced intervals of time.   This proposal builds upon previous work called “Crossing the Digital Divide” focused on integrated discovery of common information themes including precipitation in discrete data from the CUAHSI hydrologic information system and continuous data from the Unidata THREDDS data server. This project will advance that work by investigating in the first year creating new technologies for publishing and discovery of information through the Global Earth Observation System of Systems (GEOSS) Common Infrastructure, the definition of a Common Information Model for discrete and continuous data, development of shared software tools for using this Common Information Model, and extension of the concepts to similar information in the Polar, Ocean and Solid Earth Sciences.

CyberConnector Building Block Project

From the project description:

This project proposes to build an EarthCube building block, called CyberConnector, for facilitating the automatic preparation and feeding of both historic and near-real time Earth Observation (EO) customized data and on-demand derived products into Earth science models (ESMs). CyberConnector will free scientists from the laborious preparation of model inputs and release of model outputs. It will automatically process the EO data into the right products in the right form needed for ESM initialization, validation, and inter-comparison. It can support many different ESMs through its standard interfaces under a unified framework.
Unidata's main role in this project will be to assist in the installation of a Unidata IDD/LDM node at the main site at George Mason University.

Active and Ongoing Collaborations:

  • NCAR GIS Program
  • Collaboration with ESSI Labs to experiment with their brokering layer in conjunction with THREDDS Data Servers
  • UCAR wide representative to OGC Technical Committee
  • ODIP Steering Committee
  • CUAHSI Hydrological Information System Standing Committee
  • EarthCube Discrete/Continuous Building Block
  • EarchCube CyberConnector Building Block

  • Wakari Cloud-based Collaborative Python Development Environment

Areas of Reduced Commitment

  • Marine Metadata Interoperability (MMI) Project Steering Team
  • NOAA Climate Prediction and Projection Pilot Platform (NCPP)
  • AGU ESSI Focus Group Board
  • ESIN Journal Editorial Board
  • Liaison to OOI Cyberinfrastructure Project
  • U of Texas EarthCube Building Blocks project